This invention relates to an image processor that finds preferable applications in image information decoding.
MPEG (Moving Picture Image Coding Experts Group, the working group of ISO/IEC) is the international standards for moving picture data compression and extension. An MPEG decoder for reproducing moving picture data is essentially composed of a data processor. This data processor is formed of a variable length decoder (VLD), an inverse quantizer (IQ), an inverse discrete cosine transformer (IDCT), and a motion compensator (MC). In addition to these components, the MPEG decoder further requires a plurality of frame memories for motion compensation and for interlace conversion.
A key feature of the MPEG decoder is that two frames are used for motion compensation, one of which is the frame of a preceding image (an image that precedes in time) and the other of which is the frame of a following image (an image that follows in time). If motion compensation is applied to every image, this gives rise to error propagation and problems with, for example, special reproduction. To cope with these drawbacks, intra-coded pictures (I-PICTURES), predictive-coded pictures (P-PICTURES), and bidirectionally predictive-coded pictures (B-PICTURES) are introduced. The I-PICTURE is the coding type I picture and makes no reference to any other images at all. The P-PICTURE is the coding type P picture and is motion-compensated from its preceding frame. The B-PICTURE is the coding type B picture and is bidirectionally motion-compensated from both its preceding and following frames. Note that the B-PICTURE is never used as a reference frame for decoding another frame.
A manner of predicting each of the coding types I, P, and B is explained. Suppose here that input pictures' bit streams are sequentially fed to an MPEG decoder in the IO.fwdarw.P3.fwdarw.B1.fwdarw.B2 order. P3 is motion-compensated from IO. B1 is motion-compensated from IO and P3. B2 is motion-compensated from IO and P3. IO, B1, B2, and P3 are displayed in that order. In the MPEG decoder, the order of decoding and the order of display disagree. It is therefore necessary to make a change in the order within the MPEG decoder. Additionally, the decoding of B1 and B2 requires data of two frames, i.e., the data about IO and P3. This produces a need for a frame memory with a capacity of two frames for motion compensation. The MPEG decoder therefore needs a capacity of two frames for motion compensation reference.
The order of decoding by the pixel and the order of image output by the pixel are now explained. For example, in television system, even-numbered lines are first outputted and odd-numbered lines are then outputted, in other words pixels are outputted every other line from the upper left to the lower right of the screen. Whereas a field made up of the even-numbered lines only is called a top field, a field made up of the odd-numbered lines only is called a bottom field. Interlace output is executed such that the top field is first outputted from the upper left down to the lower right and the bottom field is likewise outputted from the upper left down to the lower right.
Image data are two-dimensional and, for example, two items of image data lying adjacent in space to each other are considered having a strong correlation therebetween. For the case of the interlace output, when looking at, for example, one line of the top field, a line one line up of that top field line is the line that belongs to the bottom field. In other words, although a pixel of a top field line lies spatially adjacent to a pixel of a line one line up of the top field line (i.e, a bottom field line), these pixels are apart from one another in time. If motion is violent, there may be a possibility that a stronger correlation appears between a line and another line not one line up but two lines up, since these lines lie adjacent, in time, to each other. Assuming that such a possibility may exist, the order of decoding by the pixel is roughly divided into two categories in the MPEG, namely the order of decoding in the case of the frame (FR) structure and the order of decoding in the case of the field (FD) structure.
In MPEG, the processing of decoding is executed by the basic unit known as a macro block formed of 16.times.16 pixels. One macro block is decoded from left to right. For example, the rightmost pixel of the uppermost line of an image is contained in the rightmost macro block of the image. When the decoding of the rightmost macro block is completed, data of sixteen lines is decoded. Therefore, in the MPEG, data items of sixteen lines are decoded at about the same time.
For the case of the FR structure, 1-frame data is made up of macro blocks each of which is formed of 16.times.16 pixels and each macro block is decoded individually, wherein the top and bottom fields are decoded at about the same time. As a result, the order of decoding and the order of image output disagree at all. It is therefore necessary to make a change in the order.
For the case of the FD structure, one frame is divided into a top field and a bottom field. Each of the top and bottom fields is formed of individual macro blocks each of which is made up of 16.times.16 pixels. In this case, an entire macro block belongs either to the top field only or to the bottom field only. Decoding is carried out by the macro block by field. That is, every data about the top field is decoded before the decoding of data about the bottom field starts. As a result, the order of decoding and the order of image output roughly agree; however, these orders do not agree completely because the decoding is executed by the macro block.
The top field of a frame is first outputted. This is followed by the bottom field output. When giving a look at a point where the last eight top field line output begins, the decoding of the macro block for the last sixteen lines of the frame must be completed before the point. The reason is that the values of the rightmost sixteen pixels of the last eight lines are decided only after the last macro block of the image is decoded. Therefore, both the decoding of the top field and the decoding of the bottom field must be completed at the point. Thereafter, the eight top field lines and the entire bottom field must be outputted by turns, with their data already decoded. Therefore, if data about the eight top field lines and data about the entire bottom field are not stored beforehand, they are lost before being outputted, resulting in failing to provide image outputs. In other words, this requires a frame memory with a memory capacity of about half a frame for storing data about the entire bottom field and data about the last eight top field lines.
To sum up, the motion compensation, on the one hand, requires a memory capacity of two frames, and the interlace conversion, on the other hand, requires a memory capacity of about half a frame. The total memory capacity is about 2.5 frames.
Ishiwata et al. report an MPEG decoder that employs a memory that uses one and half a frame for interlace conversion (Proceedings of the 1994 IEICE Spring Conference, C-659, March 1994). Such a memory, since it requires a capacity of one and half a frame, is expensive thereby increasing the costs of production. As pointed out above, a memory with a memory capacity of half a frame is, in principle, sufficient enough for achieving interlace conversion. Therefore, there still exists room for further improvements in reducing the memory capacity.